This PR adds a partition-addressable intermediate layout (a "partition artifact") that external IVF_PQ backends can produce and Lance can consume directly, replacing a lossy hand-off via a generic dataset.

Background

External backends such as pylance-cuvs already do the expensive work of an IVF_PQ build: they assign each row to a partition and encode PQ codes for the full dataset. Today the only way to hand that result back to Lance is to materialize it as a generic dataset; the finalizer then re-scans and re-groups the rows by partition, throwing away the partitioning the backend just computed.

Layout

An artifact is a small manifest plus a fixed number of bucketed Lance files:

artifact/
├── manifest.json
└── partitions/
    ├── bucket-00000.lance
    ├── bucket-00001.lance
    └── ...

Each row is row_id + part_id + pq_code, routed by bucket = part_id % num_buckets. The manifest records, per logical partition, the file it lives in and the (offset, num_rows) ranges inside that file:

partition 17 ->
  path: partitions/bucket-00005.lance
  ranges:
    - { offset: 1200, num_rows: 800 }
    - { offset: 4096, num_rows: 512 }

At finalize time, Lance reads only the recorded ranges for the partition it is building.

Write path

The writer is streaming and bounded in memory. For each input batch it routes rows into per-bucket in-memory buffers; when a buffer fills it is sorted by part_id, appended to the bucket file, and the covered ranges are recorded in the manifest. There is no second read/sort/rewrite pass at finish().

Changes

Rust:

• PartitionArtifactBuilder and PartitionArtifactShuffleReader
• precomputed_partition_artifact_uri plumbed into the existing vector finalization flow

Python:

• accelerator="cuvs" becomes a thin runtime delegation to the external pylance-cuvs backend. No CUDA/cuVS code lives in this tree.

Non-goals

The on-disk IVF_PQ format, finalizer semantics, and the CPU build path are unchanged. This PR only adds a new input boundary for external backends.